initializing ErrorEventsThreshold - #11955
Draft
Brett Samblanet (brettsam) wants to merge 3 commits into
Draft
Conversation
Copilot started reviewing on behalf of
Brett Samblanet (brettsam)
September 1, 2026 14:50
View session
Contributor
There was a problem hiding this comment.
Pull request overview
Adjusts RpcFunctionInvocationDispatcher to ensure ErrorEventsThreshold is initialized early (with a safe default and pre-event-subscription initialization) so worker startup/indexing errors don’t immediately exhaust the retry budget and stop the host, addressing #11871.
Changes:
- Default
ErrorEventsThresholdto3and initialize it earlier based on configured process count when possible. - Recalculate the threshold after function metadata initialization using the resolved max process count.
- Avoid setting up invocation buffers / sending load requests when starting replacement workers before metadata is available, and add regression tests for the indexing-first-error scenario.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| test/WebJobs.Script.Tests/Workers/Rpc/RpcFunctionInvocationDispatcherTests.cs | Adds coverage for default threshold behavior and worker-indexing error recovery before/after empty-metadata initialization. |
| src/Functions.Rpc.Server/Rpc/FunctionRegistration/RpcFunctionInvocationDispatcher.cs | Initializes ErrorEventsThreshold earlier and skips buffer/load setup when _functions is not yet available. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Jacob Viau (jviau)
approved these changes
Sep 1, 2026
Brett Samblanet (brettsam)
marked this pull request as draft
September 1, 2026 16:24
Member
Author
|
Ok this is more complicated than I first assumed. We've never tried to restart the process in this state and it's caused all kind of issues. Converting to draft to do more investigation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #11871
Pull request checklist
IMPORTANT: Currently, changes must be backported to the
in-procbranch to be included in Core Tools and non-Flex deployments.in-procbranch is not requiredrelease_notes.mdAdditional information
During worker indexing, a webhost language worker can publish a startup error before
RpcFunctionInvocationDispatcher.InitializeAsyncreceives function metadata. The dispatcher subscribed to worker errors in its constructor, but its restart threshold was not initialized until after the empty-function early return. This left the threshold at0, causing the first startup error to immediately exhaust the retry budget and stop the host.This change:
3and initializes it from the configured worker process count before subscribing to worker events.The regression test covers worker errors both before dispatcher initialization and after empty metadata initialization. Before the fix, it failed with
Expected: RestartWorker/Actual: StopApplication; after the fix, the configured retry budget is consumed and worker recovery is attempted. Additional coverage verifies the default threshold remains at least3.